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DESCRIPTION 

IMAGE DECODING DEVICE AND IMAGE ENCODING DEVICE 

Technical Field 

5 [OOOl] The present invention relates to an image decoding device 
which decodes a coded picture on a block-by-block basis, and more 
particularly to an image decoding device which applies filtering 
processing to the decoded blocks. 

10 Background Art 

[0002] In compression technologies for moving pictures, there are, 
as standards, H.261, H.263, and H.264 by International 
Telecommunication Union (ITU) Telecommunication Standardization 
Sector, Moving Picture Experts Group (MPEG)-l, MPEG-2, and 

15 MPEG-4 by International Organization for Standardization (ISO), 
and the like. 

[0003] In these moving picture compression technologies, one 
picture is divided into blocks and coding is performed on a 
block- by- block basis. By coding on the block- by- block basis, 

20 compression efficiency is increased, but a decoded picture causes 
block distortion more easily. In the H.264, in order to eliminate 
noise related to blocks, de-blocking filtering for eliminating the 
noise on a block-by-block basis is applied to a reference picture 
which is used in decoding or inter-picture prediction. 

25 [0004] FIG. 1 is an explanatory diagram for explaining parts to 
which de-blocking filtering is applied. 

[0005] De-blocking filtering in the H.264 is applied, as shown in FIG. 
1, to edges of a block consisting of 4 x 4 pixels in a macroblock 
consisting of 16 x 16 pixels. As shown in thick solid lines in FIG. 1 
30 (a), de-blocking filtering is applied to every edge (vertical edge) in 
a vertical direction for luma, and also as shown in thick dotted lines 
in FIG. 1 (a), de-blocking filtering is applied to some of the vertical 
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edges for chroma. In addition, as shown in thick solid lines in FIG. 
1 (b), de-blocking filtering is applied to every edge (horizontal edge) 
in a horizontal direction for luma, and also as shown in thick dotted 
lines in FIG. 1 (b), de-blocking filtering is applied to some of the 
5 horizontal edges for chroma. 

[0006] Note that, in this de-blocking filtering, eight pixels 
sandwiching the edge are used. Note also that, a determination of 
whether or not the de-blocking filtering needs to be applied depends 
on a value of a filter code (disable_deblocking_filter_Jdc). When 

10 the filter code is 0, de-blocking filtering is applied to each of the 
above-described edges of the macroblock. On the other hand, 
when the filter code is 1, de-blocking filtering is not applied to any 
edge of the macroblock. Further, when the filter code is 2, 
de-blocking filtering is applied to each of the above-described edges 

15 of a macroblock which is not positioned at a slice boundary. 

[0007] The de-blocking filtering can be divided into calculation of 
filter strength of each edge, calculation of threshold values a and 0 
which are used for the determination of filtering processing 
necessity, and the filtering processing. 

20 [0008] FIG. 2 is an explanatory diagram for explaining the 
calculation of the filter strength, the calculation of the threshold 
values, and the filtering processing. 

[0009] The filter strength of each edge is, as shown in FIG. 2 for 
example, calculated using information of a block including a pixel pO 

25 and information of a block including a pixel qO. In the meantime, 
the threshold values a and 0 which are used for the determination of 
filtering processing necessity are calculated using a quantization 
coefficient of the block including the pixel pO and a quantization 
coefficient of the block including the pixel qO. The filtering 

30 processing is applied to an edge of a block consisting of 4 x 4 pixels 
using eight pixels sandwiching the edge. Such filtering processing 
is performed, as shown in FIG. 1, on a block-by-block basis in an 
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order of the vertical edge and then the horizontal edge. At this 
point, in the filtering processing of the edge of the macroblock, 
pixels of macroblocks that are positioned on the immediately left of 
and immediately above a current macroblock and that have already 
5 been applied with de-blocking filtering, are used. Therefore, in the 
H.264, de-blocking filtering is applied to macroblocks sequentially 
from a macroblock whose address has a smaller value. Here, the 
address refers to a number which is allocated to each macroblock in 
a horizontal scanning order from a top-left corner in a picture. 

10 [OOIO] In the meantime, in the above H.264, slice groups are 
defined, and such a slice group is structured by dividing a picture 
into some regions and slices are allocated to the regions. The slice 
refers to a group of one or more macroblocks in length that are 
allocated in the slice group sequentially from a macroblock whose 

15 address has a smaller value in order not to be overlapped with one 
another. 

[0011] FIGS. 3A, 3B, and 3C are explanatory diagrams of the slice 
groups. 

[0012] In the diagrams, blocks represent macroblocks, and 
20 numbers in the blocks are IDs of slice groups to which the respective 
macroblocks belong. For example, a number "1" indicated in a 
macroblock MB represents an ID of a slice group to which the 
macroblock MB belongs. Types of the slice groups are defined as 
seven types, each of which is represented by slice_group_map_type. 
25 FIGS. 3A, 3B, and 3C show three types among them. 

[0013] FIG. 3A shows slice groups whose slice_group_map_type is 
0, and macroblocks arranged continuously in a horizontal scanning 
order form a slice group. FIG. 3B shows slice groups whose 
slice_group_map_type is 1, and this slice group forms a structure in 
30 which macroblocks belonging to the same slice group are not 
positioned side by side. FIG. 3C shows slice groups whose 
slice_group_map_type is 2, and this slice group forms a structure in 
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which, seemingly, the slice group is segmented into rectangular 
regions and such a slice group is overlapped on another slice group. 
[0014] The type of the slice group defines how the slice group is 
segmented, and an encoder allocates slice groups to a macroblock, 
5 based on the definition. Moreover, the number of the slice groups is 
regulated for each profile, and for a base line profile the number is 
one to seven, for a main profile the number is one, and for an 
extended profile the number is one to seven (see non-patent 
document 1, for example). 
10 [0015] The slice group as described above is divided into slices, 
each of which includes macroblocks. 

[0016] FIG, 3D is a diagram showing slices included in a slice group. 
[0017] For example, as shown in FIG. 3D, a slice group SG has a 
structure including a slice SI, a slice S2 and a slice S3. The slice SI 

15 has a structure including macroblocks whose addresses range from 
0 to 19, the slice S2 has a structure including macroblocks whose 
addresses range from 20 to 44, and the slice S3 has a structure 
including macroblocks whose addresses range from 45 to 64. 
[0018] The conventional image decoding device decodes slices in a 

20 order sequentially from a slice whose starting macroblock has a 
smaller address compared to other slices. For example, the image 
decoding device decodes, as shown in FIG. 3D, firstly the slice SI 
having a starting macroblock whose address is 0, next the slice S2 
having a starting macroblock whose address is 20, and then the slice 

25 S3 having a starting macroblock whose address is 45. Moreover, 
when one slice is decoded, the image decoding device decodes 
macroblocks from a starting macroblock in the slice (shaded 
macroblock in FIG. 3D) and then sequentially from a macroblock 
whose address has a smaller value. Note that a decoding order of 

30 decoding slices sequentially from a slice whose starting macroblock 
has a smaller value compared to other slices, in other words, a 
decoding order in which, when a macroblock to be decoded is 
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decoded, macroblocks positioned on the immediately left of and 
immediately above the macroblock to be decoded (the macroblocks 
having address smaller than an address of the macroblock to be 
decoded) have been already decoded, is hereinafter referred to as 
5 an address processing order. 

[0019] Then, the image decoding device applies de-blocking 
filtering to the decoded macroblocks. Here, the order of decoding 
slices and macroblocks is the above-described address processing 
order, so that the image decoding device performs the decoding and 
10 the de-blocking filtering of the macroblocks by pipelining. 

[0020] FIG. 4 is an explanatory diagram for explaining the 
pipelining. 

[0021] In general, in an exclusive hardware specialized for 
particular processing, the processing is usually divided into detail 

15 operations to form pipelines in order to perform parallel processing. 
The pipelining is a technique by which speeding up of the processing 
is aimed by performing different operations at the same time 
sequentially and executing the operations continuously without 
causing a wasted wait time of units for performing the operations. 

20 [0022] The pipelining includes, as shown in FIG. 4, pre-processing 
and post-processing. For example, each data is processed in an 
order of data 1, data 2, and then data 3. Firstly, pre-processing is 
applied to the data 1, and next post-processing of the data 1 and 
pre-processing of data 2 are performed at the same time. Next, 

25 post-processing of the data 2 and pre-processing of the data 3 are 
performed at the same time. With the above structure, it is 
possible to shorten a time of the processing and speed up the 
processing, as compared to a case where the respective 
pre-processing are performed for the data 1, the data 2, and the 

30 data, and after that the respective post-processing are performed 
for these data. Note that, in a case where the post-processing of 
the data 1 uses a result of the pre-processing of the data 2, the 
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post-processing of the data 1 and the pre-processing of the data 2 
cannot be performed in parallel, which obviously prevents the 
pipelining. 

[0023] Here, the above conventional image decoding device 
5 performs the above-described pre-processing as decoding of 
macroblocks, and performs the above-described post-processing as 
de-blocking filtering of the macroblocks. 

[0024] As described above, in the conventional image decoding 
device, the decoding order of slices and macroblocks is always the 
10 address processing order, so that the decoding and the de-blocking 
filtering of macroblocks are performed by the pipelining, in order to 
attempt to speed up a processing speed. 

Non-patent document 1: H.264/MPEG-4AVC written standards 

15 Disclosure of Invention 

[0025] However, there is a case where the order of decoding slices 
is not the address processing order. 

[0026] In the above H.264, an arbitrary slice order, which indicates 
that an order of decoding slices is not always the address processing 

20 order, is defined. Use of this arbitrary slice order is prohibited in a 
main profile, but permitted in a base line profile and an extended 
profile. In a case where the arbitrary slice order is permitted, slices 
are not always decoded sequentially from a slice whose starting 
macroblock has a smaller address compared to other slices, but 

25 sometimes decoded from a slice whose starting macroblock has a 
larger address compared to other slices. 

[0027] In such a case, when de-blocking filtering is to be applied to 
a macroblock at a slice boundary, macroblocks on the immediately 
left of and immediately above the macroblock to be processed have 
30 sometimes not yet been applied with decoding nor de-blocking 
filtering. 

[0028] Therefore, assuming that the arbitrary slice order is 
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permitted, the image decoding device needs to apply the 
de-blocking filtering to each macroblock, after all slices included in 
one picture are decoded per macroblock and thereby one decoded 
picture is constructed. This means that, in a case where the 
5 arbitrary slice order is permitted, the image decoding device cannot 
perform the decoding and the de- blocking filtering using the 
pipelining. 

[0029] FIG. 5 is a block diagram showing a structure of the image 
decoding device conforming to the arbitrary slice order in the above 
10 H.264. 

[0030] This image decoding device 800 decodes coded signal, and 
includes a variable length decoding unit 801, an inverse 
quantization unit 802, an inverse orthogonal transformation unit 
803, a memory 804, an intra-picture prediction unit 805, an adder 

15 806, a switch 807, a frame memory 808, a de-blocking filtering unit 
809, a frame memory 810, and a motion compensation unit 811. 
[0031] In the image decoding device 800 having the above 
structure, in an intra-mode for decoding coded signal of an I-picture, 
the coded signal is decoded into a quantized value by the variable 

20 length decoding unit 801, decoded into a frequency component by 
the inverse quantization unit 802, and decoded into a prediction 
error by the inverse orthogonal transformation unit 803. In 
addition, using pixels of: a macroblock on the immediately left of; a 
macroblock immediately above; a macroblock on the immediately 

25 upper left of; and a macroblock on the immediately upper right of, 
the macroblock to be processed which are stored in the memory 804, 
a predictive image is generated in the intra-picture prediction unit 
805, and the prediction error is added to the predictive image by the 
adder 806. As a result, a re-constructed image, that is the decoded 

30 macroblock, is generated. Here, in the intra-mode, the switch 807 
connects the intra-picture prediction unit 805 to the adder 806. 
The re-constructed image is stored into the memory 804 and also 
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into the frame memory 808. Then, after the re-constructed images 
of all macroblocks included in one picture are stored into the frame 
memory 808, the de-blocking filtering unit 809 applies de-blocking 
filtering for eliminating block distortion to each re-constructed 
5 image in an address order sequentially from a re-constructed image 
of a macroblock positioned at a top-left corner in the picture, and 
stores the filtered re-constructed image into the frame memory 810. 
The picture which has been applied with the de-blocking filtering and 
stored into the frame memory 810 is outputted as a decoded picture 

10 signal onto a monitor or the like. 

[0032] On the other hand, in an inter-mode for decoding coded 
signals of a P-picture and a B-picture, the coded signal is decoded 
into a quantized value by the variable length decoding unit 801, 
decoded into a frequency component by the inverse quantization 

15 unit 802, and decoded into a motion compensation error by the 
inverse orthogonal transformation unit 803. On the other hand, a 
motion compensation image is generated by the motion 
compensation unit 811, and the motion compensation error is added 
to the motion compensation image by the adder 806. As a result, a 

20 re-constructed image, that is the decoded macroblock, is generated. 
Here, in the inter-mode, the switch 807 connects the motion 
compensation unit 811 to the adder 806. The re-constructed image 
is stored into the memory 804 and also into the frame memory 808. 
Then, after re-constructed images of all macroblocks included in one 

25 picture are stored into the frame memory 808, the de-blocking 
filtering unit 809 applies de-blocking filtering for eliminating block 
distortion to each re-constructed image in an address order 
sequentially from a re-constructed image of a macroblock 
positioned at a top-left corner in the picture, and stores the filtered 

30 re-constructed image into the frame memory 810. The picture 
which has been applied with the de-blocking filtering and stored into 
the frame memory 810 is outputted as a decoded picture signal onto 
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a monitor or the like. 

[0033] Here, in a case where the filter code is 1, the de-blocking 
filtering unit 809 does not apply any processing to the 
re-constructed images and stores the re-constructed images into 
5 the frame memory 810. 

[0034] FIG. 6 is a flowchart showing a flow of processing performed 
by the image decoding device 800. 

[0035] Firstly, the image decoding device 800 decodes all coded 
macroblocks of one picture indicated by coded signal, thereby 

10 decoding the all coded macroblocks into respective re-constructed 
images (Step S822 in Loop 1). Then, after all macroblocks are 
decoded into the re-constructed images, the image decoding device 
800 applies de-blocking filtering to all re-constructed images which 
have been decoded at the Loop 1 (Step S824 in Loop 2). In other 

15 words, the image decoding device 800 applies de-blocking filtering 
to the picture on a macroblock-by-macroblock basis. 
[0036] FIG. 7 is a block diagram showing a structure of an image 
encoding device conforming to the arbitrary slice order of the H.264. 
[0037] This image encoding device 900 codes input picture signal, 

20 and includes a memory 901, an intra-picture prediction unit 902, a 
subtractor 903, a switch 904, an orthogonal transformation unit 905, 
a quantization unit 906, a variable length coding unit 907, an 
inverse quantization unit 908, an inverse orthogonal transformation 
unit 909, an adder 910, a frame memory 911, a de-blocking filtering 

25 unit 912, a frame memory 913, a motion estimation unit 914, and a 
motion compensation unit 915. 

[0038] The image encoding device 900 having the above structure 
processes a picture indicated by the input picture signal on a 
macroblock-by-macroblock basis. In an intra-mode (intra-picture 
30 coding) for coding a picture indicated by the input picture signal as 
an I-picture, using re-constructed images of: a macroblock on the 
immediately left of; a macroblock immediately above; a macroblock 
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on the immediately upper left of; a macroblock on the immediately 
upper right of, the macroblock to be processed which are stored in 
the memory 901, a predictive image is generated in the intra-picture 
prediction unit 902, and a prediction error, which is a difference 
5 between an image of the macroblock of the input picture signal and 
the predictive image, is outputted by the subtractor 903. Here, in 
the intra-mode, the switch 904 connects the intra-picture prediction 
unit 902 to the adder 903. The prediction error is transformed into 
a frequency component by the orthogonal transformation unit 905, 

10 and then compressed and coded into a quantized value in 
quantization performed by the quantization unit 906. Coding is 
applied to the quantized value by the variable length coding unit 907, 
and the coded result becomes an output. Further, the quantized 
value obtained by the quantization unit 906 is decoded into a 

15 frequency component in the inverse quantization unit 908, and 
decoded into a prediction error by the inverse orthogonal 
transformation unit 909. The prediction error is added to the 
predictive image by the adder 910. As a result, a re-constructed 
image, that is the decoded macroblock, is generated and stored into 

20 the memory 901. In addition, the re-constructed image is also 
stored into the frame memory 911. Then, after re-constructed 
images of all macroblocks included in one picture are stored into the 
frame memory 911, the de-blocking filtering unit 912 applies 
de-blocking filtering for eliminating block distortion to each 

25 re-constructed image in an address order sequentially from a 
re-constructed image of a macroblock positioned at a top-left corner 
in the picture, and stores the processed re-constructed images into 
the frame memory 913. 

[0039] On the other hand, in an inter-mode (inter-picture coding) 
30 for coding a picture indicated by the input picture signal as a 
P-picture or a B-picture, a motion vector is generated by the motion 
estimation unit 914, a predictive image is generated by the motion 
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compensation unit 915, and a motion compensation error, which is a 
difference between an image of the macroblock indicated by the 
input picture signal and the predictive image, is calculated by the 
subtractor 903. Here, in the inter-mode, the switch 904 connects 
5 the motion compensation unit 915 to the subtractor 903. 
Furthermore, the motion estimation unit 914 searches from the 
decoded picture stored in the frame memory 913 for an image area 
that has the smallest difference compare to an image of the 
macroblock to be coded. In other words, in a case of a P-picture, 

10 the motion estimation unit 914 searches from preceding reference 
picture for the above image area, while in a case of a B-picture, the 
motion estimation unit 914 searches from preceding and following 
reference pictures for the above image area. Then, the motion 
estimation unit 914 calculates, as a motion vector, an amount of 

15 motion between the searched image area and the macroblock to be 
coded indicated by the input picture signal, and then outputs the 
motion vector. Furthermore, the motion estimation unit 914 
outputs a prediction type of a macroblock, which indicates whether 
the picture to be referenced is positioned preceding, following, or 

20 preceding and following the picture to be coded. The motion 
compensation unit 915 calculates a predictive image from the 
motion vector and the prediction type that are outputted from the 
motion estimation unit 914, and outputs the prediction picture. 
[0040] The motion compensation error outputted from the 

25 subtractor 903 is transformed into a frequency component by the 
orthogonal transformation unit 905, and then compressed and 
coded into a quantized value in quantization performed by the 
quantization unit 906. Coding is applied to the quantized value by 
the variable length coding unit 907 and the coded result becomes an 

30 output. Further, the quantized value obtained by the quantization 
unit 906 is decoded into a frequency component by the inverse 
quantization unit 908, decoded into a motion compensation error by 
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the inverse orthogonal transformation unit 909, and added to the 
predictive image by the adder 910. As a result, a re-constructed 
image, that is the decoded macroblock, is generated and stored into 
the memory 901. In addition, the re-constructed image is also 
5 stored into the frame memory 911. Then, after re-constructed 
images of all macroblocks included in one picture are stored into the 
frame memory 911, the de-blocking filtering unit 912 applies 
de-blocking filtering for eliminating block distortion to each 
re-constructed image in an address order sequentially from a 
10 re-constructed image of a macroblock positioned at a top-left corner 
in the picture, and stores the processed re-constructed image into 
the frame memory 913. 

[0041] Here, in a case where the filter code is 1, the de-blocking 
filtering unit 912 does not apply any processing to the 
15 re-constructed images and stores the re-constructed images into 
the frame memory 913. 

[0042] Thus, the image decoding device 800 shown in FIG. 5 and 
the image encoding device 900 shown in FIG. 7 can perform the 
de-blocking filtering even if the decoding order is not the address 

20 processing order. 

[0043] However, the image decoding device 800 shown in FIG. 5 
and the image encoding device 900 shown in FIG. 7 have a problem, 
even if the decoding order is not the address processing order, that 
a processing speed becomes slow, since, prior to the de-blocking 

25 filtering, all macroblocks included in a picture are decoded and all 
decoded macroblocks (re-constructed images) are stored into the 
memory. 

[0044] For example, in a case where a memory storing the 
re-constructed images is an external memory and the external 
30 memory is used also for other processing besides the image 
decoding, if all re-constructed images are stored into the external 
memory in the manner as described above, an amount of data 
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stored in the external memory becomes large, so that a ratio of data 
transmitting to a whole decoding processing becomes high, which 
results in reduction of a time allocated to other processing. 
Therefore, in a case where a bus width of a bus connecting the image 
5 decoding device with the external memory is small, or where a 
processing performance of the image decoding device is low, there is 
a situation where the whole decoding processing requires more than 
a predetermined time period (an available time period for 
processing one frame), so that the image decoding cannot be 

10 performed smoothly. 

[0045] Further, prior to the de-blocking filtering, all re-constructed 
images are stored into the memory, so that the decoding 
(processing until an image is re-constructed) and the de-blocking 
filtering cannot be performed by the pipelining (parallel processing) 

15 on a macroblock-by-macroblock basis. Still further, a next picture 
to be processed is coded or decoded using a picture that has been 
applied with de-blocking filtering, so that the decoding (processing 
until an image is re-constructed) and the de-blocking filtering 
cannot be performed by the pipelining (parallel processing) even on 

20 a picture-by-picture basis. 

[0046] Thus, the present invention is conceived to solve the above 
problems, and an object of the present invention is to provide an 
image decoding device and an image encoding device which can 
eliminate block distortion and also can restrain decrease of a 

25 processing speed, even if an order of decoding blocks is changed. 

Means to Solve the Problem 

[0047] In order to achieve the above object, the image decoding 
device according to the present invention, which decodes a coded 
30 picture on a block-by-block basis, includes: a decoding unit operable 
to sequentially decode each block included in the coded picture; a 
storage unit in which each block decoded by the decoding unit is 
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stored; a filter unit operable to apply filtering processing to each 
block decoded by the decoding unit; a first executing unit operable 
to store the decoded block into the storage unit every time the block 
is decoded by the decoding unit, and to apply the filter processing to 
5 the block stored in the storage unit by the filter unit; a second 
executing unit operable to apply the filtering processing to the 
decoded block by the filter unit, every time the block is decoded by 
the decoding unit; and a switch control unit operable to switch 
between an operation performed by the first executing unit and an 

10 operation performed by the second executing unit. More 
specifically, the switch control unit switches between the operation 
performed by the first executing unit and the operation performed 
by the second executing unit, depending on an order of the blocks to 
be sequentially decoded by the decoding unit. For example, the 

15 switch control unit prohibits the operation performed by the first 
executing unit and permit the operation performed by the second 
executing unit, when the order of the blocks to be sequentially 
decoded by the decoding unit is a regular order of decoding the 
blocks continuously from a block positioned at an edge of the picture, 

20 and prohibits the operation performed by the second executing unit 
and permit the operation performed by the first executing unit, 
when the order of the blocks to be sequentially decoded by the 
decoding unit is not the regular order. 

[0048] Thereby, in a case where an order of blocks to be 
25 sequentially decoded by the decoding unit is a regular order for 
decoding the blocks continuously from a block positioned at an edge 
of a picture, namely, an address decoding order, the second 
execution unit including, for example, a switch makes a filter unit 
apply filtering processing to each block every time the block is 
30 decoded by the decoding unit. As a result, the decoding and the 
applying of the filtering processing can be performed by the 
pipelining, which results in speeding up of a processing speed more 
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compare to a case where, after each decoded block is stored into the 
storage unit, the filtering processing is applied to the block. 
Moreover, in this case, the block is not necessarily stored into the 
storage unit, so that an amount of data transmitted to the storage 
5 unit can be reduced in order to smoothly perform a whole processing. 
Moreover, in a case where an order of blocks to be sequentially 
decoded by the decoding unit is not the regular order, in other words, 
where the order is an address decoding order, the first execution 
unit including, for example, a switch makes the storage unit store 

10 each block every time the block is decoded by the decoding unit, and 
makes the filter unit apply the filtering processing to the block 
stored in the storage unit. As a result, even if the order of blocks to 
be sequentially decoded by the decoding unit is not the address 
decoding order, the filter unit can apply filtering processing to the 

15 block. Thus, according to the present invention, it is possible to 
eliminate block distortion and also to restrain decrease of a 
processing speed, even if an order of decoding each block is 
changed. 

[0049] Further, the switch control unit may switch between the 
20 operation performed by the first executing unit and the operation 
performed by the second executing unit, depending on the number 
of the blocks to be decoded by the decoding unit within a 
predetermined time period. For example, the switch control unit 
prohibits the operation performed by the first executing unit and 
25 permit the operation performed by the second executing unit, when 
the number of the blocks to be decoded by the decoding unit within 
the predetermined time period is larger than a predetermined value, 
and prohibits the operation performed by the second executing unit 
and permit the operation performed by the first executing unit, 
30 when the number of the blocks is equal to or smaller than the 
predetermined value. 

[0050], The number of blocks to be decoded within a predetermined 
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time period varies depending on profiles such as a main profile and 
a base line profile, so that a determination can be made, based on 
the number, as to whether an order of blocks to be sequentially 
decoded by the decoding unit is the regular order, or there is a 
5 possibility that the order is not the regular order. Thus, according 
to the present invention, the same effect as described above can be 
obtained by switching between an operation performed by the first 
execution unit and an operation performed by the second execution 
unit, based on the number of blocks. That is, even if the order of 

10 decoding each block is changed, it is possible to eliminate block 
distortion and also to restrain decrease of a processing speed. 
[0051] Further, the switch control unit may switch between the 
operation performed by the first executing unit and the operation 
performed by the second executing unit, based on filter information 

15 which indicates a portion that is in the block and is to be applied with 
the filtering processing, when the order of the blocks to be 
sequentially decoded by the decoding unit is not a regular order of 
decoding the blocks continuously from a block positioned at an edge 
of the picture. 

20 [0052] Thereby, even in a case, for example, where an order of 
blocks to be sequentially decoded is not the regular order, when a 
value of disable_deblocking_filter_idc as filter information is 2, in 
other words, when a slice boundary is not included in the targets to 
be applied with filtering processing, even if a plurality of slices 

25 included in one picture are decoded in any orders, the switch control 
unit makes the second execution unit execute an operation thereby 
performing the decoding and the applying of the filtering processing 
by the pipelining, so that it is possible to speed up the processing 
speed. 

30 [0053] Further, the switch control unit may switch between the 
operation performed by the first executing unit and the operation 
performed by the second executing unit, depending on a type of a 
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coded signal including the coded pictures. For example, the switch 
control unit prohibits the operation performed by the first executing 
unit and permit the operation performed by the second executing 
unit, when a determination is made, based on the type of the coded 
5 signal, that the blocks need to be decoded continuously from a block 
positioned at an edge of the picture, and prohibits the operation 
performed by the second executing unit and permit the operation 
performed by the first executing unit, when the determination is not 
able to be performed. 

10 [0054] Depending on profiles such as a main profile and a base line 
profile indicated by coded signal, there is a case where a 
determination can be made that blocks need to be decoded 
continuously from a block positioned at an edge of a picture. Thus, 
in the same manner as described for the present invention, the same 

15 effect as described above can be obtained by switching between an 
operation performed by the first execution unit and an operation 
performed by the second execution unit, depending on a type of the 
coded signal, namely, a profile. That is, even if the order of 
decoding each block is changed, it is possible to eliminate block 

20 distortion and also to restrain decrease of a processing speed. 

[0055] Further, the switch control unit may obtain an external 
signal which indicates whether or not the blocks need to be decoded 
continuously from a block positioned at an edge of the picture; 
prohibit the operation performed by the first executing unit and 

25 permit the operation performed by the second executing unit, when 
the external signal indicates that the blocks need to be decoded 
continuously; and prohibit the operation performed by the second 
executing unit and permit the operation performed by the first 
executing unit, when the external signal indicates that the blocks 

30 should not be decoded continuously. 

[0056] When the arbitrary slice order is permitted, slices are not 
always decoded continuously from a slice positioned at an edge of a 
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picture. Therefore, in the same manner as described for the 
present invention, the same effect as described above can be 
obtained by switching between an operation performed by the first 
execution unit and an operation performed by the second execution 
5 unit, depending on an external signal indicating whether or not the 
arbitrary slice order is permitted. That is, even if the order of 
decoding each block is changed, it is possible to eliminate block 
distortion and also to restrain decrease of a processing speed. 
[0057] Note that the present invention can be realized not only as 
10 the above image decoding device, but also as an image encoding 
device which includes the same structure of the image decoding 
device, as a processing method regarding the devices, a program 
regarding the devices, a storage medium which stores the program, 
and an integrated circuit. 

15 

Effect of the Invention 

[0058] An image decoding device according to the present 
invention has effects of eliminating block distortion and of 
restraining decrease of a processing speed, even if an order of 
20 decoding each block is changed. 

Brief Description of Drawings 

[0059] [FIG. 1] FIG. 1 is an explanatory diagram for explaining 
parts to which de-blocking filtering is applied. 
25 [FIG. 2] FIG. 2 is an explanatory diagram for explaining calculation 
of filter strength, calculation of threshold values, and applying of 
filtering processing. 

[FIG. 3A] FIG. 3A is an explanatory diagrams of a slice group. 
[FIG. 3B] FIG. 3B is an explanatory diagram of another slice group. 
30 [FIG. 3C] FIG. 3C is an explanatory diagram of still another slice 
group. 

[FIG. 3D] FIG. 3D is a diagram showing slices included in a slice 
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group. 

[FIG. 4] FIG. 4 is an explanatory diagram for explaining pipelining. 
[FIG. 5] FIG. 5 is a block diagram showing a structure of an image 
decoding device conforming to arbitrary slice order of H.264. 
5 [FIG. 6] FIG. 6 is a flowchart showing a flow of processing performed 
by the above image decoding device. 

[FIG. 7] FIG. 7 is a block diagram showing a structure of an image 
encoding device conforming to arbitrary slice order of H.264. 
[FIG. 8] FIG. 8 is a block diagram showing a structure of an image 
10 decoding device according to the first embodiment of the present 
invention. 

[FIG. 9] FIG. 9 is a flowchart showing a flow of processing performed 
by the image decoding device according to the first embodiment. 
[FIG. 10] FIG. 10 is a flowchart showing a flow of processing 
15 performed by an image decoding device according to a variation of 
the firsts embodiment. 

[FIG. 11] FIG. 11 is a flowchart showing a flow of processing 
performed by an image decoding device which determines a value of 
a filter code on a picture-by-picture basis. 
20 [FIG. 12] FIG. 12 is a block diagram showing a structure of an image 
encoding device according to the second embodiment of the present 
invention. 

[FIG. 13] FIG. 13 is a flowchart showing a flow of processing 
performed by an image encoding device according to the second 
25 embodiment. 

[FIG. 14] FIG. 14 is a flowchart showing a flow of processing 
performed by an image encoding device according to the first 
variation of the second embodiment. 

[FIG. 15] FIG. 15 is a flowchart showing a flow of processing 
30 performed by an image encoding device which determines a value of 
a filter code on a picture-by-picture basis. 

[FIG. 16] FIG. 16 is a flowchart showing a flow of processing 
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performed by an image encoding device according to the second 
variation of the second embodiment of the present invention. 
[FIG. 17A] FIG. 17A is an explanatory diagram of a recording 
medium which stores a program causing a computer system to 
5 execute the image decoding device and the image encoding device 
according to the present invention, (third embodiment) 
[FIG. 17B] FIG. 17B is another explanatory diagram of the recording 
medium which stores a program causing a computer system to 
execute the image decoding device and the image encoding device 

10 according to the present invention, (third embodiment) 

[FIG. 17C] FIG. 17C is still another explanatory diagram of the 
recording medium which stores a program causing a computer 
system to execute the image decoding device and the image 
encoding device according to the present invention, (third 

15 embodiment) 

[FIG. 18] FIG. 18 is a block diagram showing a whole structure of a 
content supplying system for realizing a content distribution service 
according to the fourth embodiment of the present invention. 
[FIG. 19] FIG. 19 is a diagram showing a portable telephone which 

20 uses the image encoding device and the image decoding device 
according to the fourth embodiment. 

[FIG. 20] FIG. 20 is a block diagram of the portable telephone 
according to the fourth embodiment. 

[FIG. 21] FIG. 21 is a diagram showing an example of a digital 
25 broadcasting system according to the fourth embodiment. 

Numerical References 

[0060] 100 Image decoding device 

101 Control unit 
30 102, 120 Frame memory 

103, 104, 117 Switch 

111 Variable length decoding unit 
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112 Inverse quantization unit 

113 Inverse orthogonal transformation unit 

114 Memory 

115 Intra-picture prediction unit 
5 116 Adder 

119 De-blocking filtering unit 

120 Frame memory 

121 Motion compensation unit 
200 Image encoding device 

10 201 Control unit 

202, 223 Frame memory 

203, 204, 214 Switch 
205 Setting unit 

211 Memory 

15 212 Intra-picture prediction unit 

213 Subtractor 

215 Orthogonal transformation unit 

216 Quantization unit 

217 Variable length coding unit 
20 218 Inverse quantization unit 

219 Inverse orthogonal transformation unit 

220 Adder 

222 De-blocking filtering unit 

224 Motion estimation unit 

25 225 Motion compensation unit 



Best Mode for Carrying Out the Invention 

[0061] The following describes embodiments according to the 
present invention with reference to the drawings, 
30 [0062] (First Embodiment) 

FIG. 8 is a block diagram showing a structure of an image 
decoding device 100 according to the first embodiment of the 
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present invention. 

[0063] The image decoding device 100 is an image decoding device 
which can eliminate block distortion and restrain decrease of a 
processing speed even if an order of decoding each block is changed, 
5 and the image decoding device 100 includes a variable length 
decoding unit 111, an inverse quantization unit 112, an inverse 
orthogonal transformation unit 113, a memory 114, an intra-picture 
prediction unit 115, an adder 116, a switch 117, a de-blocking 
filtering unit 119, a frame memory 120, a motion compensation unit 
10 121, a control unit 101, a frame memory 102, a switch 103, and a 
switch 104. 

[0064] The variable length decoding unit 111 performs variable 
length decoding for macroblocks, for each macroblock included in a 
coded picture indicated by coded signal, and outputs a quantized 
15 value, that is a result of the processing, to the inverse quantization 
unit 112. 

[0065] Here, the variable length decoding unit 111 obtains order 
notification information that indicates an order of decoding slices 
and macroblocks, and applies variable length decoding to each 

20 macroblock based on the order indicated in the order notification 
information. The order notification information indicates that the 
order of decoding is an address processing order or that the order of 
decoding is arbitrary. Hereinafter, the arbitrary decoding order is 
referred to as an indefinite order. 

25 [0066] Therefore, if the order notification information indicates the 
address processing order, the variable length decoding unit 111 
applies variable length decoding to slices in the coded picture 
indicated by the coded signal in an order sequentially from a slice 
whose starting macroblock has an address having a smaller value 

30 compared to other slices. Then, in each slice, the variable length 
decoding unit 111 applies variable length decoding to macroblocks 
in the slice in an order from the starting macroblock and then 
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sequentially from a macroblock whose address is smaller compared 
to other macroblocks. On the other hand, if the order notification 
information indicates the indefinite order, the variable length 
decoding unit 111 applies variable length decoding to macroblocks 
5 according to a predetermined regulation in an order that is the same 
as the above address processing order or different from the above 
address processing order. 

[0067] The inverse quantization unit 112 applies inverse 
quantization to the quantized value outputted from the variable 
10 length decoding unit 111 in order to transform the quantized value 
into a frequency component. 

[0068] The inverse orthogonal transformation unit 113 transforms, 
in an intra-mode in which the picture to be coded is an I-picture, the 
frequency component outputted from the inverse quantization unit 
15 112 into a prediction error that is image data, while in an inter- mode 
in which the picture to be coded is a P-picture or a B-picture, the 
inverse orthogonal transformation unit 113 transforms the 
frequency component into a motion compensation error that is 
image data. 

20 [0069] The switch 117 connects, in the intra-mode, the 
intra-prediction unit 115 to the adder 116, and on the other hand 
connects, in the inter-mode, the motion compensation unit 121 to 
the adder 116. 

[0070] The adder 116 adds, in the intra-mode, the prediction error 
25 to a predictive image outputted from the intra-prediction unit 115, 
thereby decoding the coded macroblock indicated by the coded 
signal into a re-constructed image. On the other hand, the adder 
116 adds, in the inter-mode, the motion compensation error to a 
motion compensation image outputted from the motion 
30 compensation unit 121, thereby decoding the coded macroblock 
indicated by the coded signal into a re-constructed image. 
[0071] The memory 114 accumulates the re-constructed image 
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outputted from the adder 116. 

[0072] The intra-picture prediction unit 115 retrieves, from a 
plurality of the re-constructed images stored in the memory 114, 
respective re-constructed images of: a macroblock positioned on 
5 the immediately left of; a macroblock positioned immediately 
above; a macroblock on the immediately upper left of; and a 
macroblock positioned on the immediately upper right of, a 
macroblock to be decoded. Then, the intra-picture prediction unit 
115 generates a predictive image of the macroblock to be decoded, 

10 using these re-constructed images. 

[0073] The control unit 101 obtains the above-described order 
notification information and controls the switch 103 and the switch 
104 based on the order notification information. 
[0074] More specifically, if the order notification information 

15 indicates the address processing order, the control unit 101 controls 
the switch 103 and the switch 104 in order to connecting the 
de-blocking filtering unit 119 directly to the adder 116. On the 
other hand, if the order notification information indicates the 
indefinite order, the control unit 101 firstly controls the switch 103 

20 in order to connect the frame memory 102 to the adder 116. The 
frame memory 102 connected to the adder 116 sequentially 
accumulates the re-constructed images outputted from the adder 
116. Then, after the frame memory 102 accumulates all 
re-constructed images of one picture, the control unit 101 controls 

25 the switch 104 in order to connect the frame memory 102 to the 
de-blocking filtering unit 119. 

[0075] When the de-blocking filtering unit 119 is connected to the 
adder 116 using the switch 103 and the switch 104, the de-blocking 
filtering unit 119 applies de-blocking filtering to each re-constructed 
30 image, every time the adder 116 outputs the re-constructed image. 
On the other hand, when the de-blocking filtering unit 119 is 
connected to the frame memory 102 using the switch 104, the 
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de-blocking filtering unit 119 applies de-blocking filtering to all 
re-constructed images of one picture that are stored in the frame 
memory 102, in an order from a macroblock positioned at a top-left 
corner in the picture and then sequentially from a macroblock whose 
5 address is smaller compare to other macroblocks. 

[0076] Then, the de-blocking filtering unit 119 sequentially stores 
the re-constructed images applied with the de-blocking filtering, 
into the frame memory 120. Thereby, in the frame memory 120, 
re-constructed images of one picture are stored as a decoded 

10 picture. Furthermore, such a decoded picture is outputted as a 
decoded picture signal onto a monitor or the like. 
[0077] Note that, when the de-blocking filtering is applied to a 
re-constructed image (macroblock) to be processed, the 
de-blocking filtering unit 119 obtains, from the frame memory 120, 

15 the re-constructed images that are positioned on the immediately 
left of and immediately above the re-constructured image and that 
have been already applied with de-blocking filtering, and uses the 
obtained re-constructed images. 

[0078] The motion compensation unit 121 obtains the decoded 
20 picture stored in the frame memory 120 as a reference picture, and 
also obtains a motion vector. Then, the motion compensation unit 
121 generates an image area that is in the reference picture and 
indicated by the motion vector, as a motion compensation image of 
the macroblock to be decoded, and outputs the motion 
25 compensation image. 

[0079] As described above, the image decoding device 100 
according to the first embodiment performs, if the order notification 
information indicates the address processing order, the decoding 
(processing until a re-constructed image is generated) and the 
30 de-blocking filtering by pipelining on a macroblock-by-macroblock 
basis. On the other hand, if the order notification information 
indicates the indefinite order, the image decoding device 100 
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decodes macroblocks, and after re-constructed images of one 
picture are stored into the frame memory 102, the image decoding 
device 100 performs de-blocking filtering. 

[0080] Here, as the above-described order notification information, 
5 profile information in the H.264 is used, for example. 

[0081] More specifically, when the profile information included in 
coded signal indicates a base line profile or an extended profile, the 
profile information indicates that an order of sequentially decoding 
macroblocks is an indefinite order. On the other hand, when the 
10 profile information indicates a main profile, the profile information 
indicates that an order of sequentially decoding macroblocks is the 
address processing order. 

[0082] In the base line profile and the extended profile the order of 
sequentially decoding macroblocks is set to the indefinite order, 

15 which is not only because the arbitrary slice order is permitted in 
these profiles but also because the number of slice groups can be set 
to range from one to seven in these profiles. This means that, 
when the number of slice groups is plural, as shown in FIG. 3B for 
example, macroblocks, which are positioned on the immediately left 

20 of and immediately above a macroblock to be decoded, sometimes 
belong to slice groups that are different from a slice group of the 
macroblock to be decoded. In such a case, in decoding of the 
macroblock to be decoded, there is a possibility that the 
macroblocks, which are positioned on the immediately left of and 

25 immediately above the macroblock to be decided, have not yet been 
decoded, so that the order of sequentially decoding the macroblocks 
becomes the indefinite order. 

[0083] FIG. 9 is a flowchart showing processing performed by the 
image decoding device 100 according to the first embodiment. 
30 [0084] The image decoding device 100 firstly determines, 
regarding a picture to be decoded, whether or not the order 
notification information indicates the address processing order 
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(Step S100). If the determination is made that the order 
notification information indicates the address processing order (Y at 
Step S100), then the image decoding device 100 applies decoding to 
each macroblock to be decoded in the address processing order 
5 (Step S102), and applies de-blocking filtering to each decoded 
macroblock (re-constructed image) (Step S104). In other words, 
the image decoding device 100 performs the decoding at the Step 
S102 and the de-blocking filtering at the Step S104 by pipelining, 
and repeats these processing for every macroblock to be decoded 
10 (Loop 2). 

[0085] On the other hand, if the determination is made that the 
order notification information does not indicate the address 
processing order, in other words, that the order notification 
information indicates an indefinite order (N at Step S100), then the 

15 image decoding device 100 decodes each macroblock to be decoded 
in the predetermined order (Step S106), and sequentially stores the 
decoded macroblocks (re-constructed images) into the frame 
memory 102 (Step S108). In this manner, the image decoding 
device 100 repeats the decoding at the Step S106 and the storing at 

20 the Step S108 for every macroblock to be decoded (Loop 3), Then, 
the image decoding device 100 applies de-blocking filtering to 
re-constructed images of one picture which are stored in the frame 
memory 102 in an order from a re-constructed image positioned at 
a top-left corner in the picture and then sequentially from a 

25 re-constructed image whose address is smaller compared to other 
re-constructed images (Step S110). 

[0086] The image decoding device 100 performs the above 
processing from the Step S100 to the Step SI 10 for all coded 
pictures indicated by the coded signal (Loop 1). 
30 [0087] As described above, according to the first embodiment, in 
the main profile in which use of a plurality of slice groups and the 
arbitrary slice order are prohibited, it is ensured that an order of 
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sequentially decoding macroblocks is an order of smaller address as 
priority (address decoding order), so that, even if all of macroblocks 
are not decoded into the re-constructed images, it is possible to 
apply de-blocking filtering to macroblocks which have been 
5 sequentially decoded into re-constructed images and not yet been 
applied with the deblocking filtering. Therefore, a ratio of data 
transmitting to a whole decoding processing can be lowered. 
Furthermore, the de-blocking filtering, which is applied to the 
sequentially decoded a re-constructed image, can be performed in 
10 parallel to the processing until a re-constructed image is generated 
(decoding). Thereby, it is possible to speed up a speed of the 
processing. 

[0088] On the other hand, in the base line profile in which use of a 
plurality of slice groups and the arbitrary slice order are permitted, 

15 it is not ensured that an order of sequentially decoding macroblocks 
is the address decoding order. In the first embodiment, even in 
such base line profile, by sequentially storing the re-constructed 
images into the frame memory 102, it is possible to apply the 
de-blocking filtering to the re-constructed images. 

20 [0089] (Variation) 

Here, a variation of the first embodiment is described. 
[0090] While the image decoding device 100 of the first 
embodiment determines, based on the order notification 
information, whether or not the decoding and the de-blocking 

25 filtering are performed by the pipelining, an image decoding devoce 
of this variation makes the determination based on the order 
notification information and the filter code 
(disable_deblocking__filter_idc). 

[0091] If the order notification information indicates the address 
30 processing order, the image decoding device according to the 
variation performs the same processing as described above, but if 
the order notification information indicates the indefinite order, the 
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image decoding device performs processing corresponding to the 
filter code. 

[0092] More specifically, if the filter code is 0, the image decoding 
device according to the variation performs the same processing as 
5 described for the case where the order notification information 
indicates the indefinite order in the first embodiment. This means 
that the control unit of the image decoding device controls the 
switch 103 in order to connect the frame memory 102 to the adder 
116, so that re-constructed images of one picture are stored into the 

10 frame memory 102. Then, the control unit controls the switch 104 
in order to connect the de-blocking filtering unit 119 to the frame 
memory 102. As a result, the de-blocking filtering unit 119 applies 
de-blocking filtering to the re-constructed images accumulated in 
the frame memory 102. Thereby, if the filter code is 0, edges of all 

15 macroblocks (re-constructed images) are applied with the 
de-blocking filtering. 

[0093] If the filter code is 1, the control unit of the image decoding 
device according to the variation controls the switch 103 and the 
switch 104 in order to connect the de-blocking filtering unit 119 

20 directly to the adder 116. Then, the control unit prevents the 
de-blocking filtering unit 119 from performing de-blocking filtering. 
As a result, the re-constructed image outputted from the adder 116 
passes through the de-blocking filtering unit 119 via the switch 103 
and the switch 104, and stored into the frame memory 120. 

25 Thereby, if the filter code is 1, no edge of all macroblocks 
(re-constructed images) is applied with the de-blocking filtering. 
[0094] If the filter code is 2, the control unit of the image decoding 
device according to the variation controls, in the same manner as 
described above, the switch 103 and the switch 104 in order to 

30 connect the de-blocking filtering unit 119 directly to the adder 116. 
Then, the control unit prevents the de-blocking filtering unit 119 
from applying de-blocking filtering to slice boundaries. As a result, 
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the de-blocking filtering unit 119 applies de-blocking filtering to 
edges (edges of a block consisting of 4 x 4 pixels in a re-constructed 
image, for example) except a slice boundary, if the re-constructed 
image (macroblock) outputted from the adder 116 is positioned at 
5 the slice boundary. On the other hand, the de-blocking filtering 
unit 119 applies de-blocking filtering to all edges of a re-constructed 
image that is outputted from the adder 116 and not positioned at a 
slice boundary, namely, to the vertical edges and horizontal edges 
shown in FIG. 1. 

10 [0095] Thereby, if the filter code is 2, for each macroblock 
(re-constructed image), the decoding and the de-blocking filtering 
are performed by the pipelining. 

[0096] FIG. 10 is a flowchart showing a flow of processing 
performed by the image decoding device according to the variation. 

15 [0097] The image decoding device according to the variation firstly 
determines, regarding a picture to be decoded, whether or not order 
notification information indicates the address processing order 
(Step S200). If the determination is made that the address 
processing order is indicated (Y at Step S200), then the image 

20 decoding device applies decoding to each macroblock to be decoded 
in the address processing order (Step S202), and applies 
de-blocking filtering to the decoded macroblock (re-constructed 
image) (Step S204). In other words, the image decoding device 
performs the decoding at the Step S202 and the de-blocking filtering 

25 at the Step S204 by pipelining, and repeats these processing for 
every macroblock to be decoded (Loop 2). 

[0098] On the other hand, if the determination is made that the 
order notification information does not indicate the address 
processing order, in other words, that the order notification 
30 information indicates an indefinite order (N at Step S200), then the 
image decoding device further determines whether or not a value of 
a filter code of each slice is 0, 1, or 2 (Step S206). Here, if the 
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value of the filter code is 0, then the image decoding device decodes 
each macroblock to be decoded in the predetermined order (Step 
S208), and sequentially stores the decoded macroblocks 
(re-constructed images) into the frame memory 102 (Step S210). 
5 In this manner, the image decoding device repeats the decoding at 
the Step S208 and the storing at the Step S210 for every macroblock 
in the slice (Loop 4). 

[0099] On the other hand, if the determination is made at the Step 
S206 that the value of the filter code is 1, then the image decoding 
10 device decodes each macroblock to be decoded in the 
predetermined order (Step S214). In this manner, the image 
decoding device repeats the decoding at the Step S214 for every 
macroblock in the slice (Loop 5). 

[0100] Moreover, if the determination is made at the Step S206 that 

15 the value of the filter code is 2, then the image decoding device 
decodes each macroblock to be decoded (Step S216), and 
determines whether or not the macroblock is positioned at a slice 
boundary (Step S218). Then, if the determination is made that the 
macroblock is positioned at a slice boundary (Y at Step S218), then 

20 the image decoding device applies de-blocking filtering to edges 
except the slice boundary in the decoded macroblock 
(re-constructed image) (Step S219), and on the other hand if the 
determination is made that the macroblock is not positioned at a 
slice boundary (N at Step S218), then the image decoding device 

25 applies de-blocking filtering to all edges in the decoded macroblock 
(re-constructed image) (Step S220). In other words, the image 
decoding device performs the decoding at the Step S216 and the 
de-blocking filtering at the Steps S219 and S220 by pipelining. In 
this manner, the image decoding device repeats the processing from 

30 the Step S216 to the Step S220 for all macroblocks in the slice (Loop 
6). 

[0101] The image decoding device performs the above processing 
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from the Step S206 to the Step S220 for all slices included in the 
picture to be decoded (Loop 3), and applies de-blocking filtering to 
each re-constructed image that is stored into the frame memory 102 
at the Step S210, in a order of smaller address as priority (Step 
5 S222). Then, the image decoding device performs the above 
processing from the Step S200 to the Step S222 for all coded 
pictures indicated by the coded signal (Loop 1). 
[0102] Note that the above has described that the image decoding 
device determines the value of the filter code for each slice, but the 
10 determination can be made for each picture. 

[0103] FIG. 11 is a flowchart showing a flow of processing 
performed by an image decoding device which determines the value 
of the filter code for each picture. 

[0104] The image decoding device firstly determines, regarding a 

15 picture to be decoded, whether or not order notification information 
indicates the address processing order (Step S250). If the 
determination is made that the address processing order is indicated 
(Y at Step S250), then the image decoding device applies decoding 
to each macroblock to be decoded in the address processing order 

20 (Step S252), and applies de-blocking filtering to the decoded 
macroblock (re-constructed image) (Step S254). In other words, 
the image decoding device performs the decoding at the Step S252 
and the de-blocking filtering at the Step S254 by pipelining, and 
repeats these processing for every macroblock of the picture to be 

25 decoded (Loop 2). 

[0105] On the other hand, if the determination is made that the 
order notification information does not indicate the address 
processing order, in other words, that the order notification 
information indicates an indefinite order (N at Step S250), then the 

30 image decoding device further determines whether or not a value of 
a filter code is 0, 1, or 2 (Step S256). Here, if the value of the filter 
code is 0, then the image decoding device decodes each macroblock 
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to be decoded in the predetermined order (Step S258), and 
sequentially stores the decoded macroblocks (re-constructed 
images) into the frame memory 102 (Step S260). In this manner, 
the image decoding device repeats the decoding at the Step S258 
5 and the storing at the Step S260 for all macroblocks in the picture to 
be decoded (Loop 3). Then, the image decoding device applies 
de-blocking filtering to re-constructed images of one picture which 
are accumulated in the frame memory 102, in an order from a 
re-constructed image positioned at a top-left corner in the picture 

10 and then sequentially from a re-constructed picture whose address 
is smaller compared to other re-constructed images (Step S262). 
[0106] On the other hand, if the determination is made at the Step 
S256 that the value of the filter code is 1, then the image decoding 
device decodes each macroblock to be decoded in the 

15 predetermined order (Step S264). In this manner, the image 
decoding device repeats the decoding at the Step S264 for all 
macroblocks in the picture to be decoded (Loop 4). 
[0107] Moreover, if the determination is made at the Step S256 that 
the value of the filter code is 2, then the image decoding device 

20 decodes each macroblock to be decoded (Step S266), and 
determines whether or not the macroblock is positioned at a slice 
boundary (Step S268). Then, if the determination is made that the 
macroblock is positioned at a slice boundary (Y at Step S268), then 
the image decoding device applies de-blocking filtering to edges 

25 except the slice boundary in the decoded macroblock 
(re-constructed image) (Step S269), and on the other hand if the 
determination is made that the macroblock is not positioned at a 
slice boundary (N at Step S268), then the image decoding device 
applies de-blocking filtering to all edges in the decoded macroblock 

30 (re-constructed image) (Step S270). In other words, the image 
decoding device performs the decoding at the Step S266 and the 
de-blocking filtering at the Steps S269 and S270 by pipelining. In 
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this manner, the image decoding device repeats the processing from 
the Step S266 to the Step S270 for all macroblocks in the picture to 
be decoded (Loop 5). 

[0108] The image decoding device performs the above processing 
5 from the Step S250 to the Step S270 for all coded pictures indicated 
by the coded signal (Loop 1). 

[0109] As described above, if it is obvious that the filter code is 
fixed for each picture and does not vary depending on each slice, the 
image decoding device performs processing without the 

10 determination of the value of the filter code of each slice. 

[0110] Note that, in the variation of the first embodiment, the 
control unit 101 controls the switch 104 in order to connect the 
de-blocking filtering unit 119 to the frame memory 102, after all 
re-constructed images of one picture have been stored into the 

15 frame memory 102, but the control unit 101 may control the switch 
104 before all re-constructed images of one picture are stored. 
[0111] More specifically, the control unit 101 controls the switch 
104, if one or more macroblocks, which have already been applied 
with de-blocking filtering and are necessary for de-blockinig filtering 

20 of the re-constructed image (the macroblock to be filtered) stored in 
the frame memory 102, are stored in the frame memory 120. 
Thereby, the control unit 101 makes the de-blocking filtering unit 
119 execute de-blocking filtering for the macroblock to be filtered. 
[0112] Moreover, even if the macroblock to be filtered does not 

25 require such filtered macroblocks, the control unit 101 controls the 
switch 104 and makes the de-blocking filtering unit 119 execute 
de-blocking filtering for the macroblock to be filtered. 
[0113] More specifically, the filtered macroblocks which are 
necessary for the macroblock to be filtered are: (1) a macroblock 

30 positioned on the immediately left of the macroblock to be filtered, 
if the macroblock to be filtered is positioned at a top edge in the 
picture; (2) macroblocks positioned immediately above and on the 
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immediately right of the macroblock to be filtered, if the macroblock 
to be filtered is positioned at a left edge in the picture; or (3) 
macroblocks positioned on the immediately left of, immediately 
above, and on the immediately upper-right of the macroblock to be 
5 filtered. Furthermore, the macroblock to be filetered which does 
not require any filtered macroblocks is a macroblock positioned at a 
top-left corner in the picture. 

[0114] Even in the above case, it is possible to sequentially apply 
the de-blocking filtering and thereby to increase the number of the 

10 de-blocking filtering which can be performed in parallel to the 
processing for generating a re-constructed image (decoding). 
[0115] Note that, in a case where decoding in the H.264 is 
macroblock adaptive frame-field decoding, the macroblock 
immediately above, the macroblock on the immediately upper-right 

15 of, and the macroblock on the immediately left of the macroblok to 
be filtered can be assumed to be a pair of macroblocks immediately 
above, a pair of macroblocks on the immediately upper-right of, and 
a pair of macroblocks on the immediately left of the macroblok to be 
filtered. Thereby the same effect as described above can be 

20 obtained. 

[0116] Note also that in the first embodiment and the variation 
thereof, as information (order notification information) indicating 
whether or not an order of macroblocks to be sequentially decoded 
is the address decoding order, information of a profile included in a 
25 stream (coded signal) is used, but it is also possible to obtain, from 
the outside, the information indicating whether or not an order of 
macroblocks to be sequentially decoded is the address decoding 
order. 

[0117] For example, in a case of a base line profile, the order of 
30 macroblocks to be sequentially decoded is generally an indefinite 
order, but, even for the base line profile, in a case where only one 
segment is broadcast, use of a plurality of slices and the arbitrary 
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slice order are not permitted, so that the order of macroblocks to be 
sequentially decoded becomes the address decoding order. 
[0118] This means that it is not possible to properly determine, 
using only the profile information, whether the order of macroblocks 
5 to be sequentially decoded is the address decoding order or the 
indefinite order. Furthermore, information indicating whether or 
not the arbitrary slice order is used does not exist in the stream, so 
that it is not possible to determine, using only the information in the 
stream, whether or not the order of macroblocks to be sequentially 

10 decoded is the address decoding order. 

[0119] Therefore, the information indicating whether or not the 
order of macroblocks to be sequentially decoded is the address 
decoding order, namely, the information indicating whether or not 
the arbitrary slice order is permitted, is obtained from the outside. 

15 Thereby, the image decoding device can properly determine whether 
or not the order of macroblocks to be sequentially decoded is the 
address decoding order. That is, the number of situations that the 
order is determined as the address decoding order is increased 
compared to the case where the determination regarding the 

20 address decoding order is made using only the profile information. 
As a result, when the determination is made that the order is the 
address decoding order, de-blocking filtering can be applied to 
macroblocks which have been sequentially decoded but not yet 
applied with de-blocking filtering, so that a ratio of data transmitting 

25 to a whole decoding processing can be lowered. In addition, the 
de-blocking filtering can be performed in parallel to the processing 
for generating a re-constructed image (decoding). Thereby, the 
processing is speeded up, eventually reducing electric power 
consumption, which can realize a long-time decoding. This is 

30 suitable for a case where a long-time use such as digital 
broadcasting. 

[0120] Note also that each of the first embodiment and the 
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variation thereof includes the frame memory 102 which stores 
re-constructed images prior to the de-blocking filtering, and the 
frame memory 913 which stores re-constructed images after the 
de-blocking filtering, but the two memories may be structured as 
5 one frame memory. 

[0121] Note also that, in the first embodiment and the variation 
thereof, the profile information is uses as the order notification 
information, and depending on the profile information the switch 
103 and the switch 104 are controlled, but it is also possible to 

10 control the switch 103 and the switch 104 depending on the number 
of macroblocks to be decoded within a predetermined time period. 
[0122] For example, in a case of a base line profile, the number of 
macroblocks to be decoded within a predetermined time period is 
smaller compared to a case of a main profile. Therefore, it is 

15 possible to determine that a profile indicated by the coded signal is 
the base line profile, if the number of the macroblocks is equal to or 
less than a predetermined value, and determine that a profile 
indicated by the coded signal is the main profile, if the number of the 
macroblocks is more than the predetermined value. 

20 [0123] Therefore, even if the switch 103 and the switch 104 is 
controlled depending on the number of macroblocks to be decoded 
within a predetermined time period, the same effect as described 
above can be obtained. 

[0124] Note also that, in the first embodiment and the variation 
25 thereof, as shown in FIG. 9 and FIG. 10, the determination whether 
or not the decoding order is the address processing order is made on 
a picture-by-picture basis, but the determination may be made on a 
slice-by-slice basis or on a sequence-by-sequence basis, and the 
determination may be made only once when the processing for 
30 coded signal starts. 

[0125] (Second Embodiment) 

FIG. 12 is a block diagram showing a structure of an image 



-37- 



encoding device 200 according to the second embodiment of the 
present invention. 

[0126] The image encoding device 200 is an image decoding device 
which can eliminate block distortion and restrain decrease of a 
5 processing speed even if an order of coding and decoding performed 
on a block-by-block basis is changed, and the image encoding device 
200 includes a memory 211, an intra-picture prediction unit 212, a 
subtractor 213, a switch 214, an orthogonal transformation unit 215, 
a quantization unit 216, a variable length coding unit 217, an 

10 inverse quantization unit 218, an inverse orthogonal transformation 
unit 219, an adder 220, a de-blocking filtering unit 222, a frame 
memory 223, a motion estimation unit 224, a motion compensation 
unit 225, a control unit 201, a frame memory 202, a switch 203, a 
switch 204, and a setting unit 205. 

15 [0127] The subtractor 213 calculates a difference between an 
image of a macroblock indicated by input picture signal and a 
predictive image obtained via the switch 214, and a prediction error, 
that is a result of the calculation, is outputted. 

[0128] The orthogonal transformation unit 215 transforms the 
20 prediction error calculated by the subtractor 213 into a frequency 
component. 

[0129] The quantization unit 216 quantizes the frequency 
component outputted from the orthogonal transformation unit 215, 
and compresses and codes the frequency component into a 

25 quantized value. 

[0130] The variable length coding unit 217 applies variable length 
coding to the quantized value outputted from the quantization unit 
216, and outputs the coded result as coded signal. 
[0131] The inverse quantization unit 218 applies inverse 

30 quantization to the quantized value outputted from the quantization 
unit 216 in order to transform the quantized value into a frequency 
component. 
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[0132] The inverse orthogonal transformation unit 219 transforms 
the frequency component outputted from the inverse quantization 
unit 218 into a prediction error that is image data. 
[0133] The adder 220 adds the prediction error outputted from the 
5 inverse orthogonal transformation unit 219 to the predictive image 
obtained via the switch 214, thereby outputting a re-constructed 
image that is the decoded macroblock. 

[0134] The memory 211 accumulates the re-constructed image 
outputted from the adder 220. 
10 [0135] The switch 214 connects the intra-picture prediction unit 
212 to the subtractor 213 in the intra-mode, and connects the 
motion compensation unit 225 to the subtractor 213 in the 
inter-mode. 

[0136] The intra-picture prediction unit 212 retrieves, from the 
15 re-constructed images stored in the memory 211, respective 
re-constructed images of: a macroblock positioned on the 
immediately left of; a macroblock positioned immediately above; a 
macroblock positioned on the immediately upper left of; a 
macroblock positioned on the immediately upper right of, a 
20 macroblock positioned to be coded. Then, the intra-picture 
prediction unit 212 generates a predictive image of the macroblock 
to be coded, using these re-constructed images. 
[0137] The setting unit 205 sets coding conditions, based on an 
operation by a user, a predetermined processing method, or the like, 
25 and notifies the coding conditions to the control unit 201. 

[0138] Here, the above-described coding conditions indicate a 
profile in the H.264, for example. 

[0139] More specifically, if the coding conditions indicate a base line 
profile or an extended profile, the coding conditions indicate that an 
30 order of macroblocks to be sequentially coded and decoded is an 
indefinite order. On the other hand, if the coding conditions 
indicate a main profile, the coding conditions indicate that an order 
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of macroblocks to be sequentially coded and decoded is the address 
processing order. 

[0140] The control unit 201 controls the switch 203 and the switch 
204, based on the coding conditions set by the setting unit 205. 
5 [0141] If the coding conditions indicate the address processing 
order, the control unit 201 controls the switch 203 and the switch 
204 in order to connecting the de-blocking filtering unit 222 directly 
to the adder 220. On the other hand, if the coding conditions 
indicate the indefinite order, the control unit 201 firstly controls the 

10 switch 203 in order to connect the frame memory 202 to the adder 
220. The frame memory 202 connected to the adder 220 
sequentially accumulates the re-constructed images outputted from 
the adder 220. Then, after the frame memory 202 accumulates all 
re-constructed images of one picture, the control unit 201 controls 

15 the switch 204 in order to connect the frame memory 202 to the 
de-blocking filtering unit 222. 

[0142] When the de-blocking filtering unit 222 is connected to the 
adder 220 using the switch 203 and the switch 204, the de-blocking 
filtering unit 222 applies de-blocking filtering to each re-constructed 

20 image, every time the adder 220 outputs the re-constructed image. 
On the other hand, when the de-blocking filtering unit 222 is 
connected to the frame memory 202 using the switch 204, the 
de-blocking filtering unit 222 applies de-blocking filtering to all 
re-constructed images of one picture that are acculumated in the 

25 frame memory 202, in an order from a macroblock positioned at a 
top-left corner in the picture and then sequentially from a 
macroblock whose address is smaller compare to other 
macroblocks. 

[0143] Then, the de-blocking filtering unit 222 stores the 
30 re-constructed images applied with the de-blocking filtering, into 
the frame memory 223. Thereby, in the frame memory 223, 
re-constructed images of one picture are stored as a decoded 
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picture. 

[0144] Note that, when the de-blocking filtering is applied to a 
re-constructed image (macroblock) to be processed, the 
de-blocking filtering unit 222 obtains, from the frame memory 223, 
5 the re-constructed images to be processed that are positioned on 
the immediately left of and immediately above the re-constructed 
image and that have been already applied with de-blocking filtering, 
and uses the obtained re-constructed images. 

[0145] The motion compensation unit 224 refers, in the inter-mode, 
10 the decoded picture stored in the frame memory 223 as a reference 
picture, thereby detecting motion of the image of the macroblock to 
be coded which is indicated by the input picture signal, and outputs 
a motion vector, that is a result of the detecting, into the motion 
compensation unit 225 and the frame memory 223. Furthermore, 
15 the motion estimation unit 914 outputs, in addition to the motion 
vector, a prediction type of the macroblock which indicates that the 
reference picture is positioned preceding, following, or preceding 
and following the picture to be coded. 

[0146] The motion compensation unit 225 obtains the decoded 
20 picture stored in the frame memory 223 as a reference picture, and 
obtains the motion vector outputted from the motion estimation unit 
224. Then, the motion compensation unit 121 generates an image 
area that is in the reference picture and indicated by the motion 
vector, as a predictive image of the macroblock to be coded, and 
25 outputs the predictive image. 

[0147] As described above, the image encoding device 200 
according to the second embodiment performs, if the coding 
conditions indicate the address processing order, the coding, the 
decoding, and the de-blocking filtering by pipelining on a 
30 macroblock-by-macroblock basis. On the other hand, if the coding 
conditions indicate the indefinite order, the image decoding device 
200 decodes each coded macroblock, and after re-constructed 
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images of one picture are stored into the frame memory 202, the 
image encoding device 200 performs de-blocking filtering. 
[0148] FIG. 13 is a flowchart showing a flow of processing 
performed by the image encoding device 200 according to the 
5 second embodiment. 

[0149] The image encoding device 200 firstly sets coding conditions 
(Step S300). Then, the image encoding device 200 determines, 
regarding a picture to be coded, whether or not the coding 
conditions indicate the address processing order (Step S302). If 

10 the determination is made that the coding conditions indicate the 
address processing order (Y at Step S302), then the image encoding 
device 200 applies coding and decoding to each macroblock to be 
coded in the address processing order (Step S304), and applies 
de-blocking filtering to each decoded macroblock (re-constructed 

15 image) (Step S306). In other words, the image encoding device 
200 performs the coding and decoding at the Step S304 and the 
de-blocking filtering at the Step S306 by pipelining, and repeats 
these processing for all macroblocks in the picture to be coded (Loop 
2). 

20 [0150] On the other hand, if the determination is made that the 
coding conditions do not indicate the address processing order, in 
other words, that the coding conditions indicate an indefinite order 
(N at Step S302), then the image encoding device 200 codes and 
decodes each macroblock to be coded in the predetermined order 

25 (Step S308), and sequentially stores the decoded macroblocks 
(re-constructed images) into the frame memory 202 (Step S310). 
In this manner, the image encoding device 200 repeats the coding 
and decoding at the Step S306 and the storing at the Step S310 for 
all macroblocks in the picture to be coded (Loop 3). Then, the 

30 image encoding device 200 applies de-blocking filtering to 
re-constructed images of one picture which are acculumated in the 
frame memory 202 in an order from a re-constructed image 
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positioned at a top-left corner in the picture and then sequentially 
from a re-constructed image whose address is smaller compared to 
other re-constructed images (Step S312). 

[0151] The image encoding device 200 performs the above 
5 processing from the Step S300 to the Step S312 for all pictures 
indicated by the coded signal (Loop 1). 
[0152] (First Variation) 

Here, the first variation of the second embodiment is 
described. 

10 [0153] While the image encoding device 200 of the second 
embodiment determines, based on only the coding conditions, 
whether or not the coding, the decoding, and the de-blocking 
filtering are performed by the pipelining, an image encoding device 
of this variation makes the determination based on the coding 

15 conditions and the filter code (disable_deblocking_filter_idc). 

[0154] If the coding conditions indicate the address processing 
order, the image encoding device according to the first variation of 
the second embodiment performs the same processing as described 
in the second embodiment, but if the coding conditions indicate the 

20 indefinite order, the image encoding device performs processing 
corresponding to the filter code. 

[0155] More specifically, if the filter code is 0, the image encoding 
device according to the first variation of the second embodiment 
performs the same processing as described for the case where the 

25 coding conditions indicate the indefinite order in the second 
embodiment. This means that the control unit of the image 
encoding device controls the switch 203 in order to connect the 
frame memory 202 to the adder 220, so that re-constructed images 
of one picture are stored into the frame memory 202. Then, the 

30 control unit controls the switch 204 in order to connect the frame 
memory 202 to the de-blocking filtering unit 222. As a result, the 
de-blocking filtering unit 222 applies de-blocking filtering to the 
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re-constructed images accumulated in the frame memory 202. 
Thereby, if the filter code is 0, edges of all macroblocks 
(re-constructed images) are applied with the de-blocking filtering. 
[0156] If the filter code is 1, the control unit of the image encoding 
5 device according to the first variation of the second embodiment 
controls the switch 203 and the switch 204 in order to connect the 
de-blocking filtering unit 222 directly to the adder 220. Then, the 
control unit prevents the de-blocking filtering unit 222 from 
performing de-blocking filtering. As a result, the re-constructed 

10 image outputted from the adder 220 passes through the de-blocking 
filtering unit 222 via the switch 203 and the switch 204, and stored 
into the frame memory 223. Thereby, if the filter code is 1, no edge 
of all macroblocks (re-constructed images) is applied with the 
de-blocking filtering. 

15 [0157] If the filter code is 2, the control unit of the image encoding 
device according to the first variation of the second embodiment 
controls, in the same manner as described above, the switch 203 
and the switch 204 in order to connect the de-blocking filtering unit 
222 directly to the adder 220. Then, the control unit prevents the 

20 de-blocking filtering unit 222 from applying de-blocking filtering to 
slice boundaries. As a result, the de-blocking filtering unit 222 
applies de-blocking filtering to edges (edges of a block consisting of 
4x4 pixels in a re-constructed image, for example) except a slice 
boundary, if the re-constructed image (macroblock) outputted from 

25 the adder 220 is positioned at the slice boundary. On the other 
hand, the de-blocking filtering unit 222 applies de-blocking filtering 
to all edges of a re-constructed image that is outputted from the 
adder 220 and not positioned at a slice boundary, namely, to the 
vertical edges and horizontal edges shown in FIG. 1. 

30 [0158] Thereby, if the filter code is 2, for each macroblock 
(re-constructed image), the coding, the decoding, and the 
de-blocking filtering are performed by the pipelining. 
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[0159] FIG. 14 is a flowchart showing a flow of processing 
performed by the image encoding device according to the first 
variation of the second embodiment, 

[0160] The image encoding device according to the first variation of 
5 the second embodiment firstly sets coding conditions (Step S400). 
Then, the image encoding device determines, regarding a picture to 
be coded, whether or not the coding conditions indicate the address 
processing order (Step S402). If the determination is made that 
the address processing order is indicated (Y at Step S402), then the 

10 image encoding device applies coding and decoding to each 
macroblock to be coded in the address processing order (Step S404), 
and applies de-blocking filtering to the decoded macroblock 
(re-constructed image) (Step S406). In other words, the image 
encoding device performs the coding and decoding at the Step S404 

15 and the de-blocking filtering at the Step S406 by pipelining, and 
repeats these processing for all macroblocks in the picture to be 
coded (Loop 2). 

[0161] On the other hand, if the determination is made that the 
coding conditions do not indicate the address processing order, in 

20 other words, that the coding conditions indicate an indefinite order 
(N at Step S402), then the image encoding device further 
determines whether or not a value of a filter code of each slice is 0, 
1, or 2 (Step S408). Here, if the determination is made that the 
value of the filter code is 0, then the image encoding device codes 

25 and decodes each macroblock to be coded in the predetermined 
order (Step S410), and sequentially stores the decoded macroblocks 
(re-constructed images) into the frame memory 202 (Step S412). 
In this manner, the image encoding device repeats the coding and 
decoding at the Step S410 and the storing at the Step S412 for all 

30 macroblocks in the slice (Loop 4). 

[0162] On the other hand, if the determination is made at the Step 
S408 that the value of the filter code is 1, then the image encoding 
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device codes and decodes each macroblock to be coded in the 
predetermined order (Step S416). In this manner, the image 
encoding device repeats the coding and decoding at the Step S416 
for all macroblocks in the slice (Loop 5). 
5 [0163] Moreover, if the determination is made at the Step S408 that 
the value of the filter code is 2, then the image encoding device 
codes and decodes each macroblock to be coded (Step S418), and 
determines whether or not the macroblock is positioned at a slice 
boundary (Step S420). Then, if the determination is made that the 

10 macroblock is positioned at a slice boundary (Y at Step S420), then 
the image encoding device applies de-blocking filtering to edges 
except the slice boundary in the decoded macroblock 
(re-constructed image) (Step S421), and on the other hand if the 
determination is made that the macroblock is not positioned at a 

15 slice boundary (N at Step S420), then the image decoding device 
applies de-blocking filtering to all edges in the decoded macroblock 
(re-constructed image) (Step S422). In other words, the image 
encoding device performs the coding and decoding at the Step S418 
and the de-blocking filtering at the Steps S421 and S422 by 

20 pipelining. In this manner, the image encoding device repeats the 
processing from the Step S418 to the Step S422 for all macroblocks 
in the slice (Loop 6). 

[0164] The image encoding device performs the above processing 
from the Step S408 to the Step S422 for all slices included in the 

25 picture to be coded (Loop 3), and applies de-blocking filtering to 
each re-constructed image that is stored into the frame memory 202 
at the Step S412, in a order of smaller address as priority (Step 
S424). Then, the image encoding device performs the above 
processing from the Step S400 to the Step S424 for all pictures 

30 indicated by the input picture signal (Loop 1). 

[0165] Note that the above has described that the image decoding 
device determines the value of the filter code for each slice, but the 
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determination can be made for each picture. 

[0166] FIG. 15 is a flowchart showing a flow of processing 
performed by an image encoding device which determines the value 
of the filter code for each picture. 
5 [0167] The image encoding device firstly sets coding conditions 
(Step S450). Then, the image encoding device firstly determines, 
regarding a picture to be coded, whether or not the coding 
conditions indicate the address processing order (Step S452). If 
the determination is made that the address processing order is 

10 indicated (Y at Step S452), then the image encoding device applies 
coding and decoding to each macroblock to be coded in the address 
processing order (Step S454), and applies de-blocking filtering to 
the decoded macroblock (re-constructed image) (Step S456). In 
other words, the image encoding device performs the coding and 

15 decoding at the Step S454 and the de-blocking filtering at the Step 
S456 by pipelining, and repeats these processing for all macroblocks 
in the picture to be coded (Loop 2). 

[0168] On the other hand, if the determination is made that the 
coding conditions do not indicate the address processing order, in 

20 other words, that the coding conditions indicate an indefinite order 
(N at Step S452), then the image encoding device further 
determines whether or not a value of a filter code is 0, 1, or 2 (Step 
S458). Here, if the value of the filter code is 0, then the image 
encoding device codes and decodes each macroblock to be coded in 

25 the predetermined order (Step S460), and sequentially stores the 
decoded macroblocks (re-constructed images) into the frame 
memory 202 (Step S462). In this manner, the image encoding 
device repeats the coding and decoding at the Step S460 and the 
storing at the Step S462 for all macroblocks in the picture to be 

30 coded (Loop 3). Then, the image encoding device applies 
de-blocking filtering to re-constructed images of one picture which 
are accumulated in the frame memory 202, in an order from a 
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re-constructed image positioned at a top-left corner in the picture 
and then sequentially from a re-constructed picture whose address 
is smaller compared to other re-constructed images (Step S464). 
[0169] On the other hand, if the determination is made at the Step 
5 S458 that the value of the filter code is 1, then the image encoding 
device codes and decodes each macroblock to be coded in the 
predetermined order (Step S466). In this manner, the image 
encoding device repeats the coding and decoding at the Step S466 
for all macroblocks in the picture to be coded (Loop 4). 

10 [0170] Moreover, if the determination is made at the Step S458 that 
the value of the filter code is 2, then the image encoding device 
codes and decodes each macroblock to be coded (Step S418), and 
determines whether or not the macroblock is positioned at a slice 
boundary (Step S470). Then, if the determination is made that the 

15 macroblock is positioned at a slice boundary (Y at Step S470), then 
the imag encoding device applies de-blocking filtering to edges 
except the slice boundary in the decoded macroblock 
(re-constructed image) (Step S471), and on the other hand if the 
determination is made that the macroblock is not positioned at a 

20 slice boundary (N at Step S470), then the image encoding device 
applies de-blocking filtering to all edges in the decoded macroblock 
(re-constructed image) (Step S472). In other words, the image 
encoding device performs the coding and decoding at the Step S468 
and the de-blocking filtering at the Steps S471 and S472 by 

25 pipelining. In this manner, the image encoding device repeats the 
processing from the Step S468 to the Step S472 for all macroblocks 
in the picture to be coded (Loop 5). 

[0171] The image encoding device performs the above processing 
from the Step S450 to the Step S472 for all pictures indicated by the 
30 input picture signal (Loop 1). 

[0172] As described above, if it is obvious that the filter code is 
fixed for each picture and does not vary depending on each slice, the 
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image encoding device performs processing without the 
determination of the value of the filter code of each slice. 
[0173] (Second Variation) 

Here, the second variation of the second embodiment is 
5 described. 

[0174] The image encoding device 200 of the second embodiment 
determines, as shown at the Step S302 of FIG. 12, whether or not 
the order is the address processing order, for each picture to be 
coded, but an image encoding device of the second variation of the 
10 second embodiment determines only once whether or not the order 
is the address processing order, not for each picture to be coded, but 
for all pictures to be coded at the same time. 

[0175] FIG. 16 is a flowchart showing a flow of processing 
performed by the image encoding device according to the second 

15 variation of the second embodiment. 

[0176] The image encoding device firstly sets coding conditions 
(Step S500). Then, the image encoding device determines whether 
or not the coding conditions indicate the address processing order 
(Step S502). If the determination is made that the address 

20 processing order is indicated (Y at Step S502), then the image 
encoding device applies coding and decoding to each macroblock 
included in a picture to be coded in the address processing order 
(Step S504), and applies de-blocking filtering to the decoded 
macroblock (re-constructed image) (Step S506). The image 

25 encoding device repeats the processing at the Step S504 and the 
processing at the Step S506 for all macroblocks in the picture to be 
coded (Loop 2). Further, the image encoding device repeats these 
processing for all pictures indicated by the input picture signal (Loop 
1). 

30 [0177] On the other hand, if the determination is made that the 
coding conditions do not indicate the address processing order, in 
other words, that the coding conditions indicate an indefinite order 
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(N at Step S502), then the image encoding device codes and 
decodes each macroblock to be coded in the predetermined order 
(Step S508), and sequentially stores the decoded macroblock 
(re-constructed image) into the frame memory 202 (Step S510). 
5 In this manner, the image encoding device repeats the processing at 
the Step S508 and the processing at the Step S510 for all 
macroblocks in the picture to be coded (Loop 4). Then, the image 
encoding device applies de-blocking filtering to re-constructed 
images of one picture which are accumulated in the frame memory 

10 202, in an order from a re-constructed image positioned at a top-left 
corner in the picture and then sequentially from a re-constructed 
picture whose address is smaller compared to other re-constructed 
images (Step S512). The image encoding device repeats the above 
processing from the Step S508 to the Step S512 for all pictures 

15 indicated by the input picture signal (Loop 3). 

[0178] Note that each of the image encoding devices of the second 
embodiment and the variations thereof include functions of each of 
the image decoding devices of the first embodiment and the 
variation thereof, so that all details described in the first 

20 embodiment and the variation thereof are applied to the second 
embodiment and the variations thereof. 

[0179] For example, each of the second embodiment and the 
variations thereof includes the frame memory 202 which stores 
re-constructed images prior to the de-blocking filtering, and the 

25 frame memory 223 which stores re-constructed images after the 
de-blocking filtering, but the two memories may be structured as 
one frame memory. 
[0180] (Third Embodiment) 

In addition, by recording a program, which realizes the image 

30 decoding device or the image encoding device described in the 
above first embodiment and the second embodiment, onto a 
recording medium such as a flexible disk, it is possible to easily 
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perform the processing described in the first embodiment and the 
second embodiment in an independent computer system. 
[0181] FIGS. 17A to 17C are explanatory diagrams of a case where 
the processing is performed by the computer system, using a 
5 flexible disk which stores a program for realizing the image decoding 
device and the image encoding device described in the first 
embodiment and the second embodiment. 

[0182] FIG. 17B shows a front view and a cross-sectional view of a 
case of the flexible disk, and a view of the flexible disk itself, and FIG. 

10 17A shows an example of a physical format of the flexible disk, as a 
recording medium body. The flexible disk FD is contained in the 
case F, and on a surface of the disk, a plurality of tracks Tr are 
formed concentrically from the outer periphery to the inner 
periphery, and each track is segmented into sixteen sectors Se in an 

15 angular direction. Therefore, in the flexible disk storing the above 
program, the image decoding method or the image coding metohd, 
as the program, is recorded in an area allocated on the above 
flexible disk FD. 

[0183] Moreover, FIG. 17C shows a structure for recording and 
20 reproducing the above program on the flexible disk FD. When the 
program is recorded onto the flexible disk FD, the image decoding 
method or the image coding method, as the program, is written from 
a computer system Cs via a flexible disk drive. Further, when the 
above image decoding method is constructed in the computer 
25 system using the program in the flexible disk, the program is read 
out from the flexible disk via the flexible disk drive and transferred 
to the computer system. 

[0184] Note that the above has described that the recording 
medium is assumed to be the flexible disk, but the above can also be 
30 performed using an optical disk. Note also that , the recording 
medium is not limited to the above mediums, but any other mediums, 
such as an IC card and a ROM cassette, can be also used, as far as 
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the mediums can record the program. 
[0185] (Fourth Embodiment) 

Furthermore, the applications of the image encoding device 
and the image decoding device described in the above embodiments, 
5 and a system using such applications are described here. 

[0186] FIG. 18 is a block diagram showing the overall configuration 
of a content supply system exlOO for realizing content distribution 
service. The area for providing communication service is divided 
into cells of desired size, and base stations exl07 to exllO which 

10 are fixed wireless stations are placed in respective cells. 

[0187] In this content supply system exlOO, various devices such 
as a computer exlll, a personal digital assistant (PDA) exll2, a 
camera exll3, a cell phone exll4 and a camera-equipped cell 
phone exll5 are connected to the Internet exlOl, via an Internet 

15 service provider exl02, a telephone network exl04 and base 
stations exl07 to exllO, for example. 

[0188] However, the content supply system exlOO is not limited to 
the combination as shown in FIG. 18, and may include a combination 
of any of these devices which are connected to each other. Also, 
20 each device may be connected directly to the telephone network 
exl04, not through the base stations exl07 to exllO which are the 
fixed wireless stations. 

[0189] The camera exll3 is a device such as a digital video camera 
capable of shooting moving images. The cell phone may be any of 

25 a cell phone of a Personal Digital Communications (PDC) system, a 
Code Division Multiple Access (CDMA) system, a Wideband-Code 
Division Multiple Access (W-CDMA) system and a Global System for 
Mobile Communications (GSM) system, a Personal Handy-phone 
System (PHS), and the like. 

30 [0190] Also, a streaming server exl03 is connected to the camera 
exll3 via the base station exl09 and the telephone network exl04, 
which realizes live distribution or the like using the camera exll3 
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based on the coded data transmitted from the user. The coding of 
the data shot by the camera may be performed by the camera exll3, 
the server for transmitting the data, or the like. Also, the moving 
image data shot by a camera exll6 may be transmitted to the 
5 streaming server exl03 via the computer exlll. The camera 
exll6 is a device such as a digital camera capable of shooting still 
and moving images. In this case, either the computer exlll or the 
camera exll6 may code the moving image data. An LSI exll7 
included in the computer exlll or the camera exll6 performs the 

10 coding processing. Note that software for coding and decoding 
images may be integrated into any type of a recording medium (such 
as a CD-ROM, a flexible disk and a hard disk) that is readable by the 
computer exlll or the like. Furthermore, the camera-equipped 
cell phone exll5 may transmit the moving image data. This 

is moving image data is the data coded by the LSI included in the cell 
phone exll5. 

[0191] In this content supply system exlOO, contents (such as a 
video of a live music performance) shot by users using the camera 
exll3, the camera exll6 or the like are coded in the same manner 

20 as in the above embodiments and transmitted to the streaming 
server exl03, while the streaming server exl03 makes stream 
distribution of the above content data to the clients at their requests. 
The clients include the computer exlll, the PDA exll2, the camera 
exll3, the cell phone exll4, and the like, capable of decoding the 

25 above-mentioned coded data. The content supply system exlOO is 
a system in which the clients can thus receive and reproduce the 
coded data, and further can receive, decode and reproduce the data 
in real time so as to realize personal broadcasting. 
[0192] When each device included in this system performs coding 

30 or decoding, the image encoding device or the image decoding 
device described in the above embodiments may be used. 
[0193] A cell phone is now described as an example thereof. 
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[0194] FIG. 19 is a diagram showing a cell phone exll5 which uses 
the image encoding device and the image decoding device as 
described in the above embodiments. The cell phone exllS has: an 
antenna ex201 for communicating radio waves with the base station 
5 exllO; a camera unit ex203 such as a CCD camera capable of 
shooting moving and still images; a display unit ex202 such as a 
liquid crystal display for displaying the data obtained by decoding 
video shot by the camera unit ex203, video received by the antenna 
ex201, or the like; a main body including a set of operation keys 

10 ex204; a voice output unit ex208 such as a speaker for outputting 
voices; a voice input unit ex205 such as a microphone for inputting 
voices; a recording medium ex207 for storing coded or decoded data, 
such as data of moving or still images shot by the camera, and data 
of text, moving images or still images of received e-mails; and a slot 

15 unit ex206 for attaching the recording medium ex207 into the cell 
phone exllS. The recording medium ex207 includes a flash 
memory element, a kind of Electrically Erasable and Programmable 
Read Only Memory (EEPROM) that is an electrically rewritable and 
erasable nonvolatile memory, in a plastic case such as an SD card. 

20 [0195] Furthermore, the cell phone exllS is described with 
reference to FIG. 20. In the cell phone exll5, a power supply 
circuit unit ex310, an operation input control unit ex304, an image 
coding unit ex312, a camera interface unit ex303, an Liquid Crystal 
Display (LCD) control unit ex302, an image decoding unit ex309, a 

25 multiplex/demultiplex unit ex308, a record/reproduce unit ex307, a 
modem circuit unit ex306 and a voice processing unit ex305, are 
connected to a main control unit ex311, and to each other, via a 
synchronous bus ex313. The main control unit ex311 is for the 
overall controlling of each unit of the main body including the display 

30 unit ex202 and the operation keys ex204. 

[0196] When a call-end key or a power key is turned ON by a user's 
operation, the power supply circuit unit ex310 supplies the 
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respective units with power from a battery pack so as to activate the 
camera-equipped digital cell phone exll5 to a ready state. 
[0197] In the cell phone exllS, under the control of the main 
control unit ex311 including a CPU, ROM, RAM and the like, the voice 
5 processing unit ex305 converts the voice signals received by the 
voice input unit ex205 in voice conversation mode into digital voice 
data, the modem circuit unit ex306 performs spread spectrum 
processing of the digital voice data, and the communication circuit 
unit ex301 performs digital-to-analog conversion and frequency 

10 transformation of the data, so as to transmit the resulting data via 
the antenna ex201. Also, in the cell phone exllS, the data 
received by the antenna ex201 in voice conversation mode is 
amplified and subjected to the frequency transformation and 
analog-to-digital conversion, the modem circuit unit ex306 

15 performs inverse spread spectrum processing of the data, and the 
voice processing unit ex305 converts it into analog voice data, so as 
to output the resulting data via the voice output unit ex208. 
[0198] Furthermore, when transmitting an e-mail in data 
communication mode, the text data of the e-mail inputted by 

20 operating the operation keys ex204 of the main body is sent out to 
the main control unit ex311 via the operation input control unit 
ex304. After the modem circuit unit ex306 performs spread 
spectrum processing of the text data and the communication circuit 
unit ex301 performs a digital-to-analog conversion and frequency 

25 transformation on the text data, the main control unit ex311 
transmits the data to the base station exllO via the antenna ex201. 
[0199] When transmitting image data in data communication mode, 
the image data shot by the camera unit ex203 is provided to the 
image coding unit ex312 via the camera interface unit ex303. 

30 When the image data is not transmitted, the image data shot by the 
camera unit ex203 can also be displayed directly on the display unit 
202 via the camera interface unit ex303 and the LCD control unit 
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ex302. 

[0200] The image coding unit ex312, including the image encoding 
device described in the present invention, compresses and codes the 
image data provided from the camera unit ex203 by the image 
5 coding method used for the image encoding device as described in 
the above embodiments so as to convert it into coded image data, 
and sends it out to the multiplex/demultiplex unit ex308. At this 
time, the cell phone exllS sends out the voices received by the 
voice input unit ex205 during the shooting by the camera unit ex203, 
10 as digital voice data, to the multiplex/demultiplex unit ex308 via the 
voice processing unit ex305. 

[0201] The multiplex/demultiplex unit ex308 multiplexes the coded 
image data provided from the image coding unit ex312 and the voice 
data provided from the voice processing unit ex305, and the modem 

15 circuit unit ex306 then performs spread spectrum processing of the 
multiplexed data obtained as the result of the processing, and the 
communication circuit unit ex301 performs digital-to-analog 
conversion and frequency transformation on the resulting data and 
transmits it via the antenna ex201. 

20 [0202] As for receiving data of a moving image file which is linked 
to a website or the like in data communication mode, the modem 
circuit unit ex306 performs inverse spread spectrum processing of 
the data received from the base station exllO via the antenna 
ex201, and sends out the multiplexed data obtained as the result of 

25 the processing to the multiplex/demultiplex unit ex308. 

[0203] In order to decode the multiplexed data received via the 
antenna ex201, the multiplex/demultiplex unit ex308 demultiplexes 
the multiplexed data into a coded bit stream of image data and a 
coded bit stream of voice data, and provides the coded image data 

30 to the image decoding unit ex309 and the voice data to the voice 
processing unit ex305, respectively, via the synchronous bus ex313. 
[0204] Next, the image decoding unit ex309, including the image 
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decoding device described in the present invention, decodes the 
coded bit stream of the image data using the decoding method 
corresponding to the coding method as described in the above 
embodiments, so as to generate reproduced moving image data, 
5 and provides this data to the display unit ex202 via the LCD control 
unit ex302, and thus moving image data included in a moving image 
file linked to a website, for instance, is displayed. At the same time, 
the voice processing unit ex305 converts the voice data into analog 
voice data, and provides this data to the voice output unit ex208, 
10 and thus voice data included in a moving image file linked to a 
website, for instance, is reproduced. 

[0205] The present invention is not limited to the above-mentioned 
system since satellite or terrestrial digital broadcasting has been in 
the news lately, and at least either the image encoding device or the 

15 image decoding device described in the above embodiments can be 
incorporated into the digital broadcasting system as shown in FIG. 
21. More specifically, a coded bit stream of video information is 
transmitted from a broadcast station ex409 to a communication or 
broadcast satellite ex410 via radio waves. Upon receipt of it, the 

20 broadcast satellite ex410 transmits radio waves for broadcasting, a 
home antenna ex406 with a satellite broadcast reception function 
receives the radio waves, and a device such as a television 
(receiver) ex401 or a Set Top Box (STB) ex407 decodes the coded bit 
stream for reproduction. The image decoding device described in 

25 the above embodiments can be implemented in a reproduction 
device ex403 for reading and decoding a coded bit stream recorded 
on a storage medium ex402 such as a CD and DVD that is a recording 
medium. In this case, the reproduced video signals are displayed 
on a monitor ex404. It is also conceived to implement the image 

30 decoding device in the set top box ex407 connected to a cable ex405 
for cable television or the antenna ex406 for satellite and/or 
terrestrial broadcasting so as to reproduce them on a monitor ex408 
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of the television. The image decoding device may be incorporated 
into the television, not in the set top box. Also, a car ex412 having 
an antenna ex411 can receive signals from the satellite ex410, the 
base station exl07 or the like, and reproduce moving images on a 
5 display device such as a car navigation system ex413 or the like in 
the car ex412. 

[0206] Furthermore, the image encoding device as described in the 
above embodiments can code image signals and record them on a 
recording medium. As a concrete example, there is a recorder 

10 ex420 such as a DVD recorder for recording image signals on a DVD 
disk ex421 and a disk recorder for recording them on a hard disk. 
They can also be recorded on an SD card ex422. If the recorder 
ex420 includes the image decoding device as described in the above 
embodiments, the image signals recorded on the DVD disk ex421 or 

15 the SD card ex422 can be reproduced for display on the monitor 
ex408. 

[0207] As for the configuration of the car navigation system ex413, 
a configuration without the camera unit ex203, the camera interface 
unit ex303 and the image coding unit ex312, out of the units as 

20 shown in FIG. 20, is conceivable. The same applies to the computer 
exlll, the television (receiver) ex401 and others. 
[0208] Moreover, three types of implementations can be conceived 
for a terminal such as the above-mentioned cell phone exll4: a 
communication terminal equipped with both an encoder and a 

25 decoder; a sending terminal equipped with an encoder only; and a 
receiving terminal equipped with a decoder only. 
[0209] Thus, the image encoding device and the image decoding 
device described in the above embodiments can be used in any of 
the above-described apparatuses and systems, and thereby the 

30 effects described in the above embodiments can be obtained. 

[0210] Note that the present invention is not limited to the above 
embodiments but various variations and modifications are possible 
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in the embodiments without departing from the scope of the present 
invention. 

[0211] Note also that functional blocks in the block diagrams (FIG. 
8 and FIG. 12) are implemented into a LSI which is an integrated 
5 circuit. These may be integrated separately, or a part or all of them 
may be integrated into a single chip. (For example, functional 
blocks except a memory may be integrated into a single chip.) 
[0212] Here, the integrated circuit is referred to as a LSI, but the 
integrated circuit can be called an IC, a system LSI, a super LSI or 

10 an ultra LSI depending on their degrees of integration. 

[0213] Note also that the technique of integrated circuit is not 
limited to the LSI, and it may be implemented as a dedicated circuit 
or a general-purpose processor. It is also possible to use a Field 
Programmable Gate Array (FPGA) that can be programmed after 

is manufacturing the LSI, or a reconfigurable processor in which 
connection and setting of circuit cells inside the LSI can be 
reconfigured. 

[0214] Furthermore, if due to the progress of semiconductor 
technologies or their derivations, new technologies for integrated 
20 circuits appear to be replaced with the LSIs, it is, of course, possible 
to use such technologies to implement the functional blocks as an 
integrated circuit. For example, biotechnology and the like can be 
applied to the above implementation. 

[0215] Note also that only a means for storing data to be coded or 
25 decoded, among these functional blocks, may be realized as another 
structure, without being integrated into the single chip. 

Industrial Applicability 

[0216] The image decoding device and the image encoding device 
30 according to the present invention have effects of eliminating block 
distortion and also restraining decrease of a processing speed, even 
if an order of decoding blocks is changed, and can be applied to, for 



-59- 



example, a portable telephone, a digital versatile disk (DVD) device, 
a personal computer, and the like. 
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